Panning in a three dimensional environment on a mobile device

ABSTRACT

This invention relates to panning in a three dimensional environment on a mobile device. In an embodiment, a computer-implemented method for navigating a virtual camera in a three dimensional environment on a mobile device having a touch screen. A user input is received indicating that an object has touched a first point on a touch screen of the mobile device and the object has been dragged to a second point on the touch screen. A first target location in the three dimensional environment is determined based on the first point on the touch screen. A second target location in the three dimensional environment is determined based on the second point on the touch screen. Finally, a three dimensional model is moved in the three dimensional environment relative to the virtual camera according to the first and second target locations.

The present application is a continuation of U.S. application Ser. No.12/546,261 having a filing date of Aug. 24, 2009, which claims thebenefit of U.S. Provisional Pat. Appl. No. 61/091,234, filed Aug. 22,2008. Applicant claims priority to and benefit of all such applicationsand incorporate all such applications herein by reference.

BACKGROUND

Field of the Invention

This invention generally relates to navigation in a three dimensionalenvironment.

Background Art

Systems exist for navigating through a three dimensional environment todisplay three dimensional data. The three dimensional environmentincludes a virtual camera that defines what three dimensional data todisplay. The virtual camera has a perspective according to its positionand orientation. By changing the perspective of the virtual camera, auser can navigate through the three dimensional environment.

Mobile devices, such as cell phones, personal digital assistants (PDAs),portable navigation devices (PNDs) and handheld game consoles, are beingmade with improved computing capabilities. Many mobile devices canaccess one or more networks, such as the Internet. Also, some mobiledevices, such as an IPHONE device available from Apple Inc., acceptinput from GPS sensors, accelerometers and touch screens. Improvedcomputing capabilities make it possible to run a wide variety ofsoftware applications on mobile devices. Despite improved computingcapabilities, many handheld mobile devices have a smalldisplay—generally less than 4 inches across. The small display may makeit difficult for a user to navigate through a three dimensionalenvironment on a mobile device.

Methods and systems are needed that improve navigation in a threedimensional environment on a mobile device.

BRIEF SUMMARY

This invention relates to panning in a three dimensional environment ona mobile device. In a first embodiment, a computer-implemented methodfor navigating a virtual camera in a three dimensional environment on amobile device having a touch screen. A user input is received indicatingthat an object has touched a first point on a touch screen of the mobiledevice and the object has been dragged to a second point on the touchscreen. A first target location in the three dimensional environment isdetermined based on the first point on the touch screen. A second targetlocation in the three dimensional environment is determined based on thesecond point on the touch screen. Finally, A three dimensional model ismoved in the three dimensional environment relative to the virtualcamera according to the first and second target locations.

In a second embodiment, a system navigates a virtual camera m a threedimensional environment on a mobile device. The system includes a touchreceiver that receives a user input indicating that an object hastouched a first point on a touch screen of the mobile device and theobject has been dragged to a second point on the touch screen. A targetmodule determines a first target location in the three dimensionalenvironment based on the first point on the touch screen and determinesa second target location in the three dimensional environment based onthe second point on the touch screen. Finally, a pan module moves athree dimensional model in the three dimensional environment relative tothe virtual camera according to the first and second target locations.

In a third embodiment, a computer-implemented method for navigating avirtual camera in a three dimensional environment on a mobile devicehaving a touch screen. A user input is received indicating that anobject has touched a first point on a screen of the mobile device andthe object has been dragged to a second point on the touch screen. Anorientation of the mobile device is received. A panning mode isdetermined from a plurality of panning modes based on the orientation ofthe model device.

Further embodiments, features, and advantages of the invention, as wellas the structure and operation of the various embodiments of theinvention are described in detail below with reference to accompanyingdrawings.

BRIEF DESCRIPTION OF THE FIGURES

The accompanying drawings, which are incorporated herein and form a partof the specification, illustrate the present invention and, togetherwith the description, further serve to explain the principles of theinvention and to enable a person skilled in the pertinent art to makeand use the invention.

FIG. 1 is a diagram illustrating a mobile device that navigates througha three dimensional environment.

FIG. 2 is a diagram illustrating a virtual camera navigating through athree dimensional environment.

FIG. 3 is a diagram illustrating a system that accepts user interfacegestures to navigate through a three dimensional environment.

FIG. 4 is a flowchart illustrating a method for angular jump navigation.

FIG. 5 is a diagram illustrating angular jump navigation on a mobiledevice.

FIGS. 6A-B are diagrams illustrating determining a target locationaccording to a position selected on a view.

FIG. 7 is a diagram illustrating an angular jump trajectory.

FIG. 8 is a flowchart illustrating a method for anchored look-aroundnavigation.

FIGS. 9A-B are diagrams illustrating anchored look-around navigation ona mobile device.

FIG. 10 is a flowchart illustrating a method for anchored helicopternavigation.

FIGS. 11A-B are diagrams illustrating anchored helicopter navigation ona mobile device.

FIG. 12 is a diagram illustrating a two finger gesture for lookingaround in a three dimensional environment on a mobile device.

FIG. 13 is a flowchart illustrating a method for navigating a virtualcamera based on an orientation of a mobile device.

FIGS. 14A-C are diagrams illustrating navigating a virtual camera basedon an orientation of a mobile device.

FIG. 15 is a flowchart illustrating a method for navigating a virtualcamera using a pinch momentum.

FIGS. 16A-C are diagrams illustrating navigating a virtual camerathrough a three dimensional environment on a mobile device using a pinchmomentum.

FIG. 17 is a flowchart illustrating a method for panning on a mobiledevice.

FIGS. 18A-B are diagrams illustrating panning through a threedimensional environment on a mobile device.

FIGS. 19A-C are diagrams illustrating different panning modes which maybe used in navigation on a mobile device.

The drawing in which an element first appears is typically indicated bythe leftmost digit or digits in the corresponding reference number. Inthe drawings, like reference numbers may indicate identical orfunctionally similar elements.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention provide for navigation m a threedimensional environment on a mobile device. In the detailed descriptionof embodiments that follows, references to “one embodiment”, “anembodiment”, “an example embodiment”, etc., indicate that the embodimentdescribed may include a particular feature, structure, orcharacteristic, but every embodiment may not necessarily include theparticular feature, structure, or characteristic. Moreover, such phrasesare not necessarily referring to the same embodiment. Further, when aparticular feature, structure, or characteristic is described inconnection with an embodiment, it is submitted that it is within theknowledge of one skilled in the art to effect such feature, structure,or characteristic in connection with other embodiments whether or notexplicitly described.

This detailed description is divided into sections. The first sectionprovides an introduction to navigation through three dimensionalenvironment on a mobile device. The second section describes a systemthat accepts user interface gestures to navigate in a three dimensionalenvironment on a mobile device. The next several sections describe theuser interface gestures in greater detail. The third section describesan angular zoom user interface gesture. The fourth section describes twoanchored navigation gestures. The fifth section describes a dual fingerlook-around gesture. The sixth section describes accelerometernavigation. The seventh section describes pitch momentum and atwo-finger touch and rotate gesture. Finally, the eight sectiondescribes panning in a three dimensional environment on a mobile device.

Introduction

This section provides an overview of navigation m a three dimensionalenvironment on a mobile device. FIG. 1 is a diagram illustrating amobile device 100 that can navigate through a three dimensionalenvironment. In embodiments, mobile device 100 may be a PDA, cell phone,handheld game console or other handheld mobile device as known to thoseof skill in the art. In an example, mobile device 100 may be an IPHONEdevice, available from Apple Inc. In another example, mobile device 100may be a device running an ANDROID platform, available from Google Inc.In other further embodiments, mobile device 100 may be a tabletcomputer, laptop computer, or other mobile device larger than a handheldmobile device but still easily carried by a user. These examples areillustrative and are not meant to limit the present invention.

Mobile device 100 may have a touch screen that accepts touch input fromthe user. The user may touch the screen with his fingers, stylus, orother means known to those skilled in the art. Mobile device 100 alsomay have an accelerometer that detects when the mobile deviceaccelerates or detects mobile device 100's orientation relative togravity. It should be noted that other devices may be user to determinemobile device 100's orientation, and this invention is not meant to belimited to an accelerometer. Further one or more accelerometers may beused. Further, mobile device 100 may have a location receiver, such as aGPS receiver, and may be connected to one or more networks such as theInternet.

Mobile device 100 has a view 102. As mentioned earlier, mobile device100 may accept touch input when a user touches view 102. Further, view102 may output images to user. In an example, mobile device 100 mayrender a three dimensional environment and may display the threedimensional environment to the user in view 102 from the perspective ofa virtual camera.

Mobile device 100 enables the user to navigate a virtual camera througha three dimensional environment. In an example, the three dimensionalenvironment may include a three dimensional model, such as a threedimensional model of the Earth. A three dimensional model of the Earthmay include satellite imagery texture mapped to three dimensionalterrain. The three dimensional model of the Earth may also includemodels of buildings and other points of interest. This example is merelyillustrative and is not meant to limit the present invention.

In response to user input, mobile device 100 may change a perspective ofthe virtual camera. Based on the virtual camera's new perspective,mobile device 100 may render a new image into view 102. Various userinterface gestures that change the virtual camera's perspective andresult in a new image are described in detail below.

FIG. 2 shows a diagram 200 illustrating a virtual camera in a threedimensional environment. Diagram 200 includes a virtual camera 202.Virtual camera 202 is directed to view a three dimensional terrain 210.Three dimensional terrain 210 may be a portion of a larger threedimensional model, such as a three dimensional model of the Earth.

As mentioned earlier, user input may cause a mobile device, such asmobile device 100 in FIG. 1, to move virtual camera 202 to a newlocation. Further, user input may cause virtual camera 202 to changeorientation, such as pitch, yaw, or roll.

In this way, user interface gestures on a mobile device cause a virtualcamera to navigate through a three dimensional environment on a mobiledevice. The various system components and details of the user interfacegestures are described below.

System

This section describes a system that navigates a virtual camera througha three dimensional environment on a mobile device in response to userinterface gestures. FIG. 3 is a diagram illustrating a system 300 thataccepts user interface gestures for navigation in a three dimensionalenvironment on a mobile device.

System 300 includes a client 302 having a user interaction module 310and a renderer module 322. User interaction module 310 includes a motionmodel 314. In general, client 302 operates as follows. User interactionmodule 310 receives user input regarding a location that a user desiresto view and, through motion model 314, constructs a view specificationdefining the virtual camera. Renderer module 322 uses the viewspecification to decide what data is to be drawn and draws the data. Ifrenderer module 322 needs to draw data that system 300 does not have,system 300 sends a request to a server for the additional data acrossone or more networks, such as the Internet, using a network interface350.

Motion model 314 constructs a view specification. The view specificationdefines the virtual camera's viewable volume within a three dimensionalspace, known as a frustum, and the position and orientation of thefrustum in the three dimensional environment. In an embodiment, thefrustum is in the shape of a truncated pyramid. The frustum has minimumand maximum view distances that can change depending on the viewingcircumstances. Thus, changing the view specification changes thegeographic data culled to the virtual camera's viewable volume. Theculled geographic data is drawn by renderer module 322.

The view specification may specify three main parameter sets for thevirtual camera: the camera tripod, the camera lens, and the camera focuscapability. The camera tripod parameter set specifies the following: thevirtual camera position (X, Y, Z coordinates); which way the virtualcamera is oriented relative to a default orientation, such as headingangle (e.g., north?, south?, in-between?); pitch (e.g., level?, down?,up?, in-between?); yaw and roll (e.g., level?, clockwise?,anti-clockwise?, in-between?). The lens parameter set specifies thefollowing: horizontal field of view (e.g., telephoto?, normal humaneye—about 55 degrees?, or wide-angle?); and vertical field of view(e.g., telephoto?, normal human eye—about 55 degrees?, or wide-angle?).The focus parameter set specifies the following: distance to thenear-clip plane (e.g., how close to the “lens” can the virtual camerasee, where objects closer are not drawn); and distance to the far-clipplane (e.g., how far from the lens can the virtual camera see, whereobjects further are not drawn). As used herein “moving the virtualcamera” includes zooming the virtual camera as well as translating thevirtual camera.

To construct a view specification, user interaction module 310 receivesuser input. Client 302 has various mechanisms for receiving input. Forexample, client 302 may receive input using sensors including a touchreceiver 340, an accelerometer 342, and a location module 344. Each ofthe sensors will now be described in turn.

Touch receiver 340 may be any type of touch receiver that accepts inputfrom a touch screen. Touch receiver 340 may receive touch input on aview such as the view 102 in FIG. 1. The touch input received mayinclude a position that the user touched as defined by an X and Ycoordinate on the screen. The user may touch the screen with a finger,stylus, or other object. Touch receiver 340 may be able to receivemultiple touches simultaneously if, for example, the user selectsmultiple locations on the screen. The screen may detect touches usingany technology known in the art including, but not limited to,resistive, capacitive, infrared, surface acoustic wave, strain gauge,optical imaging, acoustic pulse recognition, frustrated total internalreflection, and diffused laser imaging technologies.

Accelerometer 342 may be any type of accelerometer as known to thoseskilled in the art. Accelerometer 342 may be able to detect when themobile device moves. Accelerometer 342 also may be able to detect theorientation of a mobile device relative to gravity.

Location receiver 344 detects the location of the mobile device.Location receiver 344 may detect a location of a mobile device from, forexample, a GPS receiver. A GPS receiver determines a location of themobile device using signals from GPS satellites. In other examples,location receiver 344 may detect location of mobile device by, forexample, collecting information from nearby cell towers and wi-fihotspots. Location receiver 344 may use information from cell towers,wi-fi hotspots, and GPS satellites together to determine the location ofthe mobile device quickly and accurately.

As mentioned earlier, user interaction module 310 includes variousmodules that change the perspective of the virtual camera as defined bythe view specification. User interaction module 310 includes a momentummodule 316, an angular jump module 312, a navigation module 318, ananchor module 320, a pan module 348, and a target module 346. Each ofthese modules is described below.

The modules in user interaction module 310 may change a virtual camera'sperspective according to a target location. A target location may bedetermined by a target module 346. In an embodiment, target module 346may extend a ray from a focal point of the virtual camera. The targetlocation may be an intersection of the ray with a three dimensionalmodel, such as a three dimensional model of the Earth. The ray may beextended according to a position on the view selected by a user.Alternatively, the ray may be extended through a center of the viewfrustum of the virtual camera. The operation of target module 346 isdescribed in more detail with respect to FIGS. 6A-B.

One module that uses target module 346 is angular jump module 312. Inresponse to a user selecting a feature in the three dimensionalenvironment, angular jump module 312 moves the virtual camera toward thefeature. In an embodiment, touch receiver 340 receives a user inputindicating that a user has selected a position of a view. In an example,a user may select a position on the view and initiate an angular jump bydouble tapping on the position. Based on the position selected by theuser, target module 346 determines a target location. Using the targetlocation, angular jump module 312 moves the virtual camera. Angular jumpmodule 312 may move the virtual camera toward the target location andmay rotate the virtual camera toward the target location. As the virtualcamera moves, angular jump module 312 may change the virtual camera'sroll to simulate an airplane banking. Angular jump module 312 may orientthe virtual camera such that the target location appears approximatelyat the center of the view. To orient the virtual camera, angular jumpmodule 312 may change pitch or yaw values of the virtual camera. In thisway, a user can double tap on a screen with one hand and easily navigatethe virtual camera towards the target. Further, the smooth transition ofthe virtual camera to its new location may create a pleasing effect to auser.

Anchor module 320 moves the virtual camera in response to other userinterface gestures. In an embodiment, anchor module 320 is called whentouch receiver 340 receives a two finger touch with one fingerstationary and the other finger in motion. The relative initialpositions of the stationary and moving fingers may activate one of twonavigation modes—an anchored look-around mode or an anchored helicoptermode. In an embodiment, the anchored look-around mode is activated whenthe initial position of the first stationary finger is below the initialposition of the second finger. The anchored helicopter mode is activatedwhen the initial position of the first stationary finger is above theinitial position of the second finger. The anchored look-around mode beexecuted by a look-around module 326, and the anchored helicopter modemay be executed by a helicopter module 324.

Look-around module 326 changes an orientation of the virtual cameraaccording to movement of the second finger. Touch receiver 340 mayreceive the direction of the second finger's movement and send thedirection to look-around module 326. Based on the direction, look-aroundmodule 326 may rotate the virtual camera along different axes.Look-around module 326 may change a yaw of the virtual camera whenfinger moves toward the left or right of the mobile device. Similarly,look-around module 326 may change a pitch of the virtual camera when thefinger moves toward the top or bottom of the mobile device. Theoperation of look-around module 326 is described in more detail withrespect to FIGS. 8 and 9A-B. In an embodiment, look-around module 326also may change an orientation of the virtual camera in response tomovement of two fingers. This embodiment is described with respect toFIG. 12.

Helicopter module 324 moves the virtual camera when the position of thestationary finger is initially below the moving finger. In anembodiment, when touch receiver 340 receives a two finger touch with thestationary finger below the moving finger, target module 346 maydetermine a target location. The target location may be determined byextending a ray based on the position of the stationary finger.Alternatively, the target location may be determined by extending a raythrough a center of the virtual camera's view frustum. Determining atarget location is described in more detail later with respect to FIGS.6A-B.

Touch receiver 340 may send a direction of the moving finger tohelicopter module 324. Based on the direction of the moving finger,helicopter module 324 may move the virtual camera in differentdirections, keeping a distance between the target location and theposition of the virtual camera approximately constant. Helicopter module324 may allow for small changes in the distance. For example, newterrain may be streamed into the client that causes the distance tochange.

Helicopter module 324 may extend a ray upwards from the target locationdetermined by target module 346. When the direction of the moving fingeris towards the top or bottom of the mobile device, helicopter module 324may change a tilt angle relative to the ray. Changing the tilt angle maymove the virtual camera up or down. When the direction of the movingfinger is towards the left or right of the mobile device, helicoptermodule 324 may change an azimuth angle relative to the ray. Changing anazimuth angle may move the virtual camera around the target locationwhile maintaining a constant elevation. Further, when the direction ofthe moving finger has components on both axes of the mobile device,helicopter module 324 may change both the tilt and azimuth angles. Inthis way, helicopter module 324 enables a user to navigate easily arounda target location. Helicopter module may also move the virtual camerawhen two fingers rotate on a screen of a mobile device as described forFIG. 16C.

In an embodiment, helicopter module 324 also may change a distancebetween the target location and the virtual camera. For example, thevirtual camera may move into or away from the target location. Forexample, movement of the initially stationary finger may result intranslating the virtual camera in towards or away from the target.

In an embodiment, helicopter module 324 may change an azimuth anglewhile allowing navigation module 318 to change a tilt angle based on anorientation of the mobile device relative to gravity. The operation ofhelicopter module 324 is described in more detail with respect to FIG.10 and FIGS. 11A-B.

Navigation module 318 orients and positions the virtual camera in thethree dimensional environment according to orientation and positioninformation received from accelerometer 342 and location receiver 344.Navigation module 318 includes an accelerometer navigation module 330.In an embodiment, accelerometer 342 receives an orientation of themobile device relative to gravity. Based on the orientation of themobile device, accelerometer navigation module 330 changes a position ororientation of the virtual camera. Based on the orientation of themobile device, accelerometer navigation module 330 may change a pitch ofthe virtual camera, causing the virtual camera to look up and down.Alternatively, accelerometer navigation module 330 may change a tilt ofthe virtual camera relative to a target location, causing the virtualcamera to move up or down.

Location receiver 344 may receive a heading value of the mobile device.For example, location receiver 344 may receive the cardinal direction(north, east, south, west) that the mobile device faces. Based on theheading value, navigation module 318 may orient the virtual camera inthe direction of the mobile device. Also, location receiver 344 mayreceive a location value of the mobile device. For example, locationreceiver 344 may receive may receive a latitude, longitude and altitudeof the mobile device. Based on the location of the mobile device,navigation module 318 may position a virtual camera in the threedimensional environment. The three dimensional environment may include athree dimensional model of the Earth. In this way, navigation module 318may position and orient the virtual camera in the virtual Earth tocorrespond to the position and orientation of the mobile device in thereal Earth. Navigation module 318 may continually update the positionand orientation of the virtual camera to track the mobile device. Theoperation of navigation module 318 is described in more detail withrespect to FIG. 13 and FIGS. 14A-B.

Each of angular jump module 312, momentum module 316, accelerometernavigation module 330, look-around module 326, and helicopter module 324accept user interface gestures to move the virtual camera. Each of thosemodules may coordinate with momentum module 316 to continue the motionof the virtual camera after the user interface is gesture is complete.Momentum module 316 may gradually decelerate the motion after thegesture is complete. In this way, momentum module 316 simulates thevirtual camera having a momentum and simulates the virtual camera beingsubjected to friction, such as air resistance.

As described above, anchor module 316 navigates a virtual camera whentouch receiver 340 receives a two finger touch with one fingerstationary and the other in motion. According to a further feature, whenboth fingers are in motion, momentum module 316 also may navigate thevirtual camera. A two finger touch with both fingers in motion issometimes described herein as a pinch gesture with the fingers eithermoving away from each other or towards each other. Momentum module 316may determine a speed that the fingers relative to each other. Based onthe finger speed, momentum module 316 may determine a speed of thevirtual camera and may move the virtual camera at the determined speed.Moving the fingers towards each other may cause the virtual camera tomove forward, whereas moving the fingers away from each other may causethe virtual camera to move backwards. Momentum module 316 may simulateair resistance and consequently may reduce the speed of the virtualcamera gradually.

Alternatively, the virtual camera may remain stationary and a threedimensional model, such as a three dimensional model of the Earth, maymove according to the finger speed. Momentum module 316 may rotate amodel of the Earth at an angular velocity determined according to afinger speed. The operation of momentum module 316 is described in moredetail with respect to FIG. 15 and FIG. 16A-B.

A three dimensional model, such as a three dimensional model of theEarth, may also be rotated by pan module 348. In an embodiment, touchreceiver 340 may receive a user input indicating that a user has toucheda first position on a view of the mobile device and moved his finger toa second position on the view (a touch-and-drag gesture). Based on thefirst and second positions, target module 346 may determine first andsecond points in the three dimensional environment. Based on the firstand second points, pan module 348 may move the three dimensional modelrelative to the virtual camera. This movement may be referred to hereinas “panning.” In an example, pan module 348 may move the threedimensional model by determining a rotation axis on the threedimensional model and rotating the three dimensional model around therotation axis.

In an embodiment, the operation of pan module 348 may change accordingto the orientation of the virtual camera. As mentioned earlier, theorientation of the virtual camera may be determined according to anorientation of the mobile device relative to gravity. In an example,when the virtual camera faces the ground, the user may pan in anydirection. However, when the virtual camera faces the horizon, the usermay pan only forward and backwards. Finger movements to the left andright instead may result in the virtual camera looking to the left orright. The operation of pan module 348 is described in greater detailwith respect to FIG. 17, FIGS. 18A-B, and FIGS. 19A-C.

Each of the components of system 300 may be implemented in hardware,software, firmware, or any combination thereof.

In the following sections, the operation of angular jump module 312,target module 346, anchor module 320, momentum module 316, navigationmodule 318 and pan module 348 is described in greater detail.

Angular Jump

This section describes a method for angular jump navigation with respectto FIGS. 4-5, 6A-B, and 7. Angular jump navigation enables a user tonavigate easily and intuitively in a three dimensional environment on amobile device. In general, in response to a user double tapping on alocation, the method navigates a virtual camera toward a location andangles the virtual camera toward the location.

FIG. 4 is a flowchart illustrating a method 400 for angular jumpnavigation. Method 400 begins with receiving a user input indicatingthat a user has double tapped on a location of a view at step 402. Step402 is illustrated in FIG. 5. FIG. 5 shows a diagram 500 illustratingangular jump navigation on a mobile device. Diagram 500 shows mobiledevice 100 with view 102. A user double taps at a location 504. Angularjump navigation navigates along a trajectory 502 as is described in theremaining steps of method 400.

Based on a location of the tap location received in step 402, a targetlocation is determined at step 404. Determining a target location isillustrated in FIGS. 6A-B. FIG. 6A shows a diagram 600 illustratingextending a screen ray to determine a target location. Diagram 600 showsa virtual camera with a focal point 602. The virtual camera has a focallength 606 and a viewport 604. On viewport 604, point 610 corresponds toa point selected by a user on a view of the mobile device. From focalpoint 602, a ray 612 is extended through point 610. Ray 612 intersectswith a three dimensional model 616 to determine a target location 614.In this way, target location 614 is determined based on the pointselected (e.g., double tapped) by the user.

While being easy for user, double tapping a view with a finger can beimprecise. Mobile devices tend to have small views (handheld mobiledevices, for example, may have views generally not larger than 4inches). As result, a finger touch may occupy a substantial portion ofthe view. When the user selects a position that is close to the horizon,the screen ray may be nearly tangential to the three dimensional model.Small changes in the position of the wide finger may result in largechanges in the target location. As result, angular jump navigation maybe unstable.

To deal with potential instability, the user selection may be damped asillustrated in FIG. 6B. FIG. 6B shows a diagram 650 with a virtualcamera having focal point 602, focal length 606 and viewport 604. A userselects a point on the view close to a horizon. The point selected bythe user corresponds to a point 652 on viewport 604. A ray 654 extendsfrom focal point 602 through point 652 on viewport 604. Ray 654intersects with a concave virtual surface 658 at a point 656. Point 656may be projected onto three dimensional model 660 to determine a targetlocation. By intersecting a screen ray with a virtual surface, theuser's selection is damped, thus improving stability. Diagram 650 showsone method for damping a user selection, but other methods may be usedas are known to those of skill in the art.

Referring back to FIG. 4, once a target location is determined, thevirtual camera moves toward the target location at step 406. As thevirtual camera moves toward the target location, the virtual camerarotates toward the target location at step 408. Steps 406 and 408 areillustrated in FIG. 7.

FIG. 7 shows a diagram 700 illustrating an angular jump trajectory.Diagram 700 shows a virtual camera at an initial position 702. Thevirtual camera moves along a trajectory 706. The virtual camera maystart with an initial forward velocity vector. As the virtual cameracontinues on trajectory 706, the virtual camera rotates towards a target708. Rotating towards a target 708 may include changing a pitch or yawof the virtual camera. As the virtual camera continues on trajectory706, the virtual camera may slow down, coming to rest at a position 704facing the target 708. When the virtual camera comes to rest, the target708 may appear at approximately the center of the view. The approximatecenter of the view may not be the exact center as small offsets from thecenter are allowed.

As the virtual camera moves along trajectory 706, the virtual camera mayroll. The roll may simulate an aircraft-like turn toward thedestination. The virtual camera may start trajectory 706 with no roll.The virtual camera's roll may increase as it moves along trajectory 706and may attain the largest amount of roll midway through trajectory 706.Then, the virtual camera's roll may decrease returning to zero roll whenthe virtual camera reaches its final position 704.

In this way, angular jump navigation enables a user to easily navigatetowards a target location in a three dimensional environment.Additionally, by determining the target location based on a double touchgesture, the user can navigate towards the location with only one hand.This is useful because often users have one hand holding the mobiledevice, leaving only one hand free to navigate in the three dimensionalenvironment.

Anchored Navigation

With one free hand to navigate, several user interface gestures may usetwo fingers. This section describes two user interface gestures usingtwo fingers—anchored look-around and anchored helicopter. Each userinterface gesture has one finger initially stationary with the othermoving. The stationary finger may touch the screen before the movingfinger. The initial relative position of the stationary and movingfingers may determine whether the user enters an anchored look-aroundnavigation mode or an anchored helicopter navigation mode.

FIG. 8 is a flowchart illustrating a method 800 for anchored look-aroundnavigation. Method 800 begins by receiving a user input for a two fingertouch on a view of the mobile device at step 802. One of the two fingersis in motion and the direction of motion (e.g. a motion vector) of thesecond finger is received at step 804. The two finger touch isillustrated in FIG. 9A.

FIG. 9A shows a diagram 900. Diagram 900 shows mobile device 100 withview 102. A user has touched view 102 with a finger 902 and a finger904. Finger 902 is initially stationary and finger 904 is initially inmotion. Finger 902 may touch the screen at least a certain amount timebefore finger 904. As result of the relative position of finger 902 and904, the user enters an anchored look-around navigation mode. In anembodiment, the user enters anchored navigation mode when the fingerinitially moving (finger 904) is above the finger that is initiallystationary (finger 902).

Referring back to FIG. 8, once the movement of the second finger isreceived, an orientation of the virtual camera is changed according tothe movement of the second finger. How the virtual camera's orientationis changed is illustrated in FIG. 9B.

FIG. 9B shows a diagram 950 illustrating a virtual camera looking aroundin a three dimensional environment. Diagram 950 shows three dimensionalterrain 210 and virtual camera 202. By changing its pitch, camera 202may look up and down as show by an arrow 952. By changing its yaw,camera 202 may look left and right as show by an arrow 954.

In FIG. 9A, when a user moves finger 904 to the left or right, as shownby arrows 906 and 908, the virtual camera's yaw may change to look leftor right accordingly. Similarly, when a user moves finger 904 up ordown, as shown by arrows 906 and 908, the virtual camera's pitch maychange to look up or down accordingly. When a user moves his finger at adiagonal, as shown by arrow 920, both a pitch and a yaw of the virtualcamera may change. The pitch and yaw may change according to thecomponents of the motion vector along the axes of the mobile device. Inthis way, by moving a finger, a user can cause the virtual camera tolook-around, viewing the three dimensional environment from differentperspectives.

In an alternative embodiment, the virtual camera may look to the leftand right based on the user input, while looking up and down based on anorientation of a mobile device. An orientation of the mobile devicerelative to gravity may be received from an accelerometer of the mobiledevice. A pitch of the virtual camera may be changed according to theorientation of the mobile device. In this way, the user can look up anddown by angling the mobile device up and down.

In an embodiment, an axis of the virtual camera may be determined basedon the position of the first, stationary finger. In an example, a targetlocation may be determined based on the position of the stationaryfinger. The axis is the line connecting the virtual camera and thetarget location. In this embodiment, movement of the second fingercauses the virtual camera to rotate about the axis.

As mentioned earlier, a user enters an anchored look-around mode whenthe stationary finger is below the moving finger. However, when thestationary finger is above the moving finger, the user may enter ananchored helicopter mode. Anchored helicopter mode is described withrespect to FIGS. 10 and 11A-B.

FIG. 10 is a flowchart illustrating a method 1000 for anchoredhelicopter navigation. Method 1000 begins by receiving a user input fora two finger touch on a view of the mobile device at step 1002. One ofthe two fingers is in motion and the direction of motion (e.g. a motionvector) of the second finger is received at step 1004. The two fingertouch is illustrated in FIG. 11A.

FIG. 11A shows a diagram 1100. Diagram 1100 shows mobile device 100 withview 102. A user has touched view 102 with a finger 1102 and a finger1104. Finger 1102 is initially stationary and finger 1104 is initiallyin motion. A user may touch view 102 with a finger 1102 a certain timeprior to touching the view with a finger 1104. As result of the relativeposition of finger 1102 and 1104, the user enters an anchored helicoptermode. As mentioned above, the user may enter anchored helicopter modewhen the finger initially moving (finger 1104) is below the finger thatis initially stationary (finger 1102).

Referring back to FIG. 10, after receiving user input, a target locationis determined at step 1004. In an embodiment, the target location may bedetermined based on the position of the first, stationary finger. Thetarget location may be determined by extending a screen ray as describedin FIG. 6A. Further, the screen ray may be damped as described withrespect to FIG. 6B. Alternatively, the target location may be determinedby extending a ray through a center of the virtual camera's viewfrustum. The ray may intersect with a three dimensional model at atarget location. These examples are illustrative, and other methods ofdetermining a target location may be used as are known to those of skillin the art.

Once a target location is determined, a tilt or azimuth value relativeto the target location is changed according to the movement of thesecond finger at step 1008. Step 1008 is illustrated in FIG. 11B.

FIG. 11 B shows a diagram 1150 illustrating anchored helicopternavigation. Diagram 1150 shows virtual camera 202 directed towards threedimensional terrain 210. A ray 1160 is extended to determine a target1158 as described for step 1006. From target 1158, a vector 1162directed upwards is determined. Relative to vector 1162, virtual camera202 has a tilt angle 1156 and an azimuth angle 1154. Changing tilt angle1156 causes virtual camera 202 to move up or down, and changing azimuthangle 1154 causes virtual camera 202 to orbit around target 1158 at aconstant elevation. In an embodiment, changing tilt angle 1156 andazimuth angle 1154 does not change the distance between virtual camera202 and target 1158. In this way, changing tilt angle 1156 and azimuthangle 1154 navigates the virtual camera around target 1158 while stayingequidistant to target 1158.

In FIG. 11A, when a user moves finger 1104 left or right, as shown byarrows 1108 and 1106, an azimuth angle changes causing virtual camera202 to orbit around the target 1158 at a constant elevation. Similarly,when a user moves finger 904 up or down, as shown by arrows 1110 and1112, a tilt angle may change moving the virtual camera up and downrelative to target 1158. When a user moves his finger at a diagonal, asshown by arrow 1116, both the tilt and azimuth angles may change. Thetilt and azimuth values may change according the components of themotion vector along the axes of the mobile device. In this way, bymoving a finger, a user can cause the virtual camera to move around atarget location, viewing a target location from different perspectives.An orientation of the virtual camera may also change such that thevirtual camera continues to face the target.

In an example, a user may move finger 1104 down and to the right. Inthis example, both a tilt and azimuth value relative to a targetlocation may increase in response to the finger movement. As the tiltvalue increases the virtual camera moves down towards the elevation ofthe target location. Meanwhile, the increasing azimuth value causes thevirtual camera to rotate around the target location. While the virtualcamera is moving, the virtual camera may remain oriented toward thetarget location. In this way, a user can easily view a feature in thethree dimensional environment from different perspectives.

In an embodiment, the distance between the virtual camera and the targetlocation may also change. For example, the virtual camera may swoop intothe target by moving the virtual camera into a target while changing atilt or azimuth value. Also, the virtual camera can move away from thetarget while changing a tilt or azimuth value.

In an embodiment, moving finger 1104 left or right may change an azimuthangle, while a tilt angle is determined according to an orientation ofthe mobile device. An orientation of the mobile device relative togravity may be received from an accelerometer of the mobile device.Based on the orientation of the mobile device, the tilt angle isdetermined. In this way, the user may move the virtual camera up anddown by moving the mobile device up and down.

For instance, a user holding the mobile device and viewing a display maymove the device relative to the ground. As the device moves to face theground, the virtual camera may move above the target and face downtoward the target. As the device moves to perpendicular to the ground,the virtual camera may move to the target's elevation and view thetarget from a ground-level view.

Dual Finger Look-Around Navigation

As mentioned earlier, a user may cause a virtual camera to look aroundby moving one finger and keeping another stationary. This sectiondescribes another gesture that may cause a virtual camera to lookaround. The gesture described in this section includes two fingerstouching the display. In general, two fingers move in approximately thesame direction by approximately the same distance and the virtual cameramoves according to the finger movement.

FIG. 12 shows a diagram 1200 illustrating a two finger gesture forlooking around in a three dimensional environment on a mobile device.Diagram 1200 shows mobile device 100 with view 102. Touching view 102are fingers 1202 and 1204. With the user touching view 102, user movesfingers 1202 and 1204 on view 102 as shown by vectors 1206 and 1208.Vectors 1206 and 1208 represent the direction and distance that a usermoves fingers 1202 and 1204.

Vectors 1206 and 1208 may be in approximately in the same direction.Vectors 1206 and 1208 need not be exactly parallel. A small anglebetween directions 1206 and 1208 may be allowed up to a threshold.Similarly, vectors 1206 and 1208 may have approximately the same length.A small difference in the length of vectors 1206 and 1208 may be allowedup to a threshold.

Based on the direction and distance that the user moves fingers 1202 and1204, a virtual camera's orientation changes. Fingers 1202 and 1204 havemoved slightly different directions and distances then the direction anddistance values may be combined to determine an aggregate vector. In anexample, the direction and distance values of vectors 1206 and 1208 maybe averaged to determine the aggregate vector. Here a vector isdescribed but any type of motion data may be used.

In response to movement of fingers 1202 and 1204, the virtual camera maychange its orientation as illustrated in FIG. 9B. FIG. 9B shows adiagram 950 with three dimensional terrain 210 and virtual camera 202.Diagram 950 shows three dimensional terrain 210 and virtual camera 202.When the vector of finger movement is to the left or right on the mobiledevice, the virtual camera's yaw may change. Changing the virtualcamera's yaw causes the camera to look left or right as show by arrow954. Similarly, when the vector of finger movement is up or down on themobile device, the virtual camera's pitch may change. Changing thevirtual camera's pitch causes the camera to look to up or down as shownby arrow 956.

When a user moves his finger at a diagonal, both a pitch and a yaw ofthe virtual camera may change. The pitch and yaw may change according tothe components of the vector of the finger movement along the axes ofthe mobile device. In this way, by moving two fingers, a user can causethe virtual camera to look-around, viewing the three dimensionalenvironment from different perspectives.

In an alternative embodiment, the virtual camera may look to the leftand right based on the user input, while looking up and down based on anorientation of a mobile device. An orientation of the mobile devicerelative to gravity may be received from an accelerometer of the mobiledevice. A pitch of the virtual camera may be changed according to theorientation of the mobile device. In this way, the user can look up anddown by angling the mobile device up and down. The orientation of themobile device may be determined by an accelerometer. The next sectiondescribes accelerometer navigation in greater detail.

Accelerometer Navigation

This section describes navigating a virtual camera with an accelerometerin greater detail. FIG. 13 is a flowchart illustrating a method 1300 fornavigating a virtual camera based on an orientation of a mobile device.

Method 1300 begins with enabling accelerometer navigation at step 1302.Accelerometer navigation may be enabled, for example, when a user makesa setting change to turn it on or at startup if a default setting is setfor accelerometer navigation. In another example, entering a navigationmode such as anchored navigation or look-around navigation may enableaccelerometer navigation. Also, accelerometer navigation may be enabledwhen a change in orientation of the mobile device exceeds a threshold.This way minor changes in orientation do not unintentionally change theperspective of the virtual camera. Also, the accelerometer navigationmay be enabled when an orientation of the mobile device relative togravity exceeds a threshold. If an orientation of the mobile devicerelative to gravity is below a threshold, the orientation may be in a“dead zone”.

Once accelerometer navigation is enabled, an orientation of the mobiledevice is determined at step 1304. In an embodiment, an accelerometerdetermines the direction of gravity and an orientation of the mobiledevice relative to gravity. Based on the orientation of the mobiledevice, the virtual camera's position or orientation is changed at step1306. Steps 1304 and 1306 are illustrated in FIGS. 14A-C. Further, theaccelerometer readings may be damped.

FIGS. 14A-C each show a mobile device with a different orientation. FIG.14A shows a profile of a mobile device 1402 facing the ground. Supposethat an orientation of mobile device 1402 is defined relative to avector normal to the plane of the screen of mobile device 1402. Anaccelerometer of the mobile device detects that gravity is facingstraight down. In other words, gravity is parallel to the orientation ofmobile device 1402. As result, the virtual camera is oriented straightdown at a three dimensional model, such as a three dimensional model ofthe Earth. With the virtual camera facing the ground, the virtual cameramay capture an image 1404 of the ground.

FIG. 14B shows a profile of a mobile device 1422 at an angle relativethe ground. An accelerometer of the mobile device detects that gravityhas an angle 1426 relative to the orientation of the mobile device. Asresult, the virtual camera's pitch may be set to angle 1426. Orientingthe virtual camera at angle 1426, an image captured by the virtualcamera and displayed to the user may appear as an image 1424.

Alternatively, the virtual camera's pitch may be determined based onangle 1426. In an embodiment, a range of angles of the mobile device mayinterpolate smoothly to a range of angles of the virtual camera. Theinterpolation may be a linear interpolation. In an example, suppose therange of angles of the mobile device is 30 degrees to 90 degrees. Thatrange interpolate to a range of angles of the virtual camera of 0degrees to 90 degrees. In that example, if a user holds the device at 60degrees, an angle of the virtual camera may be set to 45 degrees. Thisexample is merely illustrative.

FIG. 14C shows a profile of a mobile device 1432 normal to the ground.An accelerometer of the mobile device detects that gravity has an angle1436 relative to the mobile device. As result, the virtual camera'spitch may be set to angle 1436. Orienting the virtual camera at angle1436, an image captured by the virtual camera and displayed to the usermay appear as an image 1434 facing the horizon.

As illustrated in FIGS. 14A-C, as the user changes an orientation of themobile device, an orientation of the virtual camera changes. Thus, asthe user directs the mobile device toward the horizon, the virtualcamera looks toward the horizon. As the user directs the mobile devicetowards the sky, the virtual camera looks toward the sky. Finally, asthe user directs the mobile device towards the ground, the virtualcamera looks toward the ground.

In addition to changing an orientation of the virtual camera, a positionof the virtual camera may also change according to an orientation of amobile device. In an embodiment, a target location and a tilt angle maybe determined as described with respect to FIGS. 11A-B. As theorientation of the mobile device changes, a tilt angle of the virtualcamera relative to a target location may change.

In this way, a user can navigate through a three dimensional environmentby changing an orientation of a mobile device.

Pinch Momentum

The anchored navigation section discussed a two finger gesture with onefinger initially stationary and the other initially in motion. Thissection describes a two finger gesture with both fingers initially inmotion. The two finger gesture may be referred to as a pinch and isdescribed with respect to FIG. 15 and FIGS. 16A-B. A pinch also may bedistinguished from anchored navigation by the timing of the first andsecond finger touches. For example, when a time between first and secondfingers is above a threshold, an anchored navigation mode may beactivated. When a time between first and second fingers is above athreshold, the virtual camera may be moved with a pinch momentum. In analternative embodiment, the anchored navigation mode may be activatedwhen the time is below a threshold, and the virtual camera may be movedwith a pinch momentum when the time is above a threshold.

FIG. 15 is a flowchart illustrating a method 1500 for navigating avirtual camera using a pinch. Method 1500 begins by receiving an inputfor a user pinch on the view at 1502. A user pinch is described isillustrated in FIG. 16A.

FIG. 16A shows a diagram 1600 illustrating a pinch gesture on a mobiledevice. Diagram 1600 shows mobile device 100 with view 102. A user hastouched view with fingers 1604 and 1602. Both fingers are in motion andtheir relative motion is a speed of the pinch determined in step 1504.Moving fingers 1604 and 1602 apart as shown with arrows 1612 and 1614may result in a positive pinch speed, whereas moving fingers 1604 and1602 together as shown with arrows 1624 and 1622 may result in anegative pinch speed.

Based on the pinch speed determined in step 1504, a virtual camera speedis determined at step 1506. The virtual camera speed may be positive(forward) if the pinch speed is positive, and the virtual camera speedmay be negative (reverse) if the pinch speed is negative. In an example,the virtual camera speed may be linearly interpolated from the pinchspeed. This is just an illustrative example and this not meant to limitthe present invention.

At step 1508, the virtual camera accelerates to the speed determined atstep 1506. At step 1510, the virtual camera may decelerate gradually. Todecelerate the virtual camera, a momentum of the virtual camera may besimulated, and the virtual camera may be exposed to a simulated airresistance. Steps 1508 and 1510 are illustrated in FIG. 16B.

FIG. 16B shows a diagram 1650 illustrating a virtual camera subjected toa pinch momentum. Diagram 1650 shows a virtual camera starting at aposition 1652 and ending at a position 1654. Diagram 1650 shows thevirtual camera at several points in time t₀, t₁, t₂, t₃, t₄, and t₅. Astime passed, the virtual camera decelerates.

In another embodiment, both fingers need not be initially in motion. Oneor both fingers could be initially stationary. Further, a pinch maytranslate the virtual camera or cause a virtual camera to zoom withoutany momentum. In that embodiment, the virtual camera zooms or translatesaccording a distance or speed of the pinch. When the pinch gesture iscompleted, the virtual camera may stop zooming or translating.

In an embodiment, the virtual camera may be translated in a straightline. Alternatively, the virtual camera may stay stationary and thethree dimensional model may move. In an example, the three dimensionalmodel may rotate. This motion of the three dimensional model relative tothe virtual camera may be referred to as “panning”.

In another embodiment, the virtual camera is both zoomed (or translated)and rotated. The rotation of the camera is based on the angle betweenthe two fingers, and the zoom is based on the distance between the twofingers. These two actions can be done simultaneously. Neither fingerneeds to be anchored for this gesture, but either finger may beanchored. This embodiment is illustrated in FIG. 16C.

In FIG. 16C, finger 1 and finger 2 are in contact with surface at thesame time. Further, finger 1 and finger 2 may be in motion at the sametime. Rotating finger 1 and finger 2 as illustrated by arrows 1671 and1673 may result in rotating the camera around a target point. The targetpoint may be determined by extending a screen ray as described for FIGS.6A-B. In examples, the screen ray may be determined based on thelocation of one the fingers, such as the first finger to touch thescreen. Alternatively, the screen ray may be determined based on amidpoint between the fingers. In this way, the target point is notcovered by one of the user's fingers on the display.

Once the target point is determined, the camera may rotate around thetarget point. In one embodiment, the camera may rotate around the targetpoint by changing an azimuth value as described for FIG. 11B. In thisway, the camera may helicopter around a target point, viewing the targetfrom different perspectives.

In one embodiment, an “invisible” line may be determined connectingfinger 1 and finger 2. When a user rotates finger 1 and 2 as illustratedby arrows 1671 and 1673, an angle between the invisible line and thedisplay of the mobile device changes as well. When the angle between theinvisible line and the display of the mobile device changes, an azimuthangle relative to a target point may also change. In one embodiment, theazimuth angle may change by the same amount, or approximately the sameamount, as the angle between the invisible line and the display of themobile device. In this way, when a user rotates two fingers on thedisplay of the mobile device by 360 degrees, the virtual camerahelicopters 360 degrees around the target point.

Further, changing a distance between finger 1 and finger 2, asillustrated with arrow 1679, may change a range of virtual camera, e.g.,by zooming or translating the virtual camera. In one example, aninvisible line connecting finger 1 and 2 is determined as describedabove. When the invisible line decreases in length, the camera may moveaway from a target point. Similarly, when the invisible line increasesin length, the camera may move toward a target point, or vice versa.Changing the range is described above with respect to FIGS. 16A-B.Further, a momentum may be applied to continue the gesture as discussedabove. A speed of either the rotation, the zoom, or both may diminishgradually after removal of fingers based on a speed at end of gesture.

In one example operation, the user may rotate finger 1 and 2counter-clockwise by 90 degrees and may move finger 1 and 2 apart. Inthat example, the virtual camera may helicopter around the target pointby 90 degrees counter-clockwise and may translate closer to the targetpoint. In another example operation, the user may rotate finger 1 and 2clockwise by 45 degrees and may move finger 1 and 2 closer together. Inthat example, the virtual camera may helicopter around the target pointby 45 degrees clockwise and may translate away from the target point.

By zooming and rotating in a single user interface gesture, embodimentsenable a user to navigate easily around a target point and to view atarget from different perspectives.

Angular Panning

This section describes panning a virtual camera through a threedimensional environment on a mobile device. In general, a user pans byselecting a position on the view of the mobile device with a finger.Based on the selected position, a target location is determined. As theuser drags his finger, the position of the three dimensional modelrelative to the virtual camera moves to follow the target location. Thismay be referred to as a touch-and-drag gesture. In an embodiment, thethree dimensional model rotates to follow the user's finger in responseto the touch-and-drag gesture.

FIG. 17 is a flowchart illustrating a method 1700 for panning on amobile device. Method 1700 begins at step 1702 with receiving a firstand second position selected by a user of a mobile device. Selecting thefirst and second positions is illustrated in FIG. 18A. Each of the firstand second position may be defined by an X and Y coordinates on theview. FIG. 18A shows a diagram 1800 illustrating panning on a mobiledevice. Diagram 1800 shows mobile device 100 with view 102. A usertouches a position 1802 with his finger and drags his finger to a newposition 1804.

Based on position 1802 and position 1804, first and second target pointsare determined at step 1704. The first and second target points may bedetermined with rays as described with respect to FIG. 6A-B. If the rayis nearly tangential to the three dimensional model, the target pointmay need to be damped as described with respect to FIG. 6B. Each targetpoint may be defined by, for example, a latitude, longitude, andaltitude. Altitude (as the term is meant here) may be the distance fromthe target point to a center of the three dimensional model. In anembodiment, the first target point is determined by intersecting a raywith the three dimensional model and the second target point isdetermined by intersecting a ray with a virtual surface sphere.Determining the target points is illustrated in FIG. 18B.

FIG. 18B shows a diagram 1800 with virtual camera 202 facing threedimensional terrain 210. As mentioned earlier, three dimensional terrain210 may be a portion of a three dimensional model. In an embodiment, thefirst target point (target point 1854) may be determined by extending aray 1852 to intersect with the three dimensional model at threedimensional terrain 210. Based on target point 1854, a virtual spheresurface 1862 is determined. Virtual sphere surface 1862 may have acenter at the center of the three dimensional model and may be tangenttarget point 1854. By intersecting a second ray 1864 with virtual spheresurface 1862, a target point 1856 is determined. Alternatively, avirtual surface may not be used and the second target point may bedetermined by intersecting a ray with the three dimensional model. Thesetwo points, target point 1854 and target point 1856, form a geocentricvector relative to the center of the three dimensional model.

Referring back to FIG. 17, once the target points are determined, arotation axis is determined at step 1706. To compute a rotation axis,cross product between the two target points may be determined. Referringto FIG. 18B, the two target points may be defined by two vectors V1′ andV1. The rotation axis is computed by taking the cross product betweenV1′ and V1 (V1′×V1). Once the rotation axis is determined, the threedimensional model is rotated at step 1708. The three dimensional modelis rotated by an angle a determined by computing the dot product betweenthe two vectors V1′ and V1 (α=V1′·V1). A rotation matrix is computedbased on the angle α and the rotation axis. Finally, the threedimensional model is rotated based on the rotation matrix.

Upon the completion of the panning motion, the last screen spaceposition of the finger may be recorded. Further, the panning motion maycontinue after the user gesture is completed. This gives the feeling tothe user that he is spinning a globe. The speed of rotation may decreasegradually to simulate friction.

In this way, a target grabbed by a user with his finger follows theuser's finger movements. To the user, it may feel as if he is touchingthe planet and manipulating it. Due to the size of the view, the firstand second positions of the finger cannot be too far apart. This limitsthe speed at which a user can pan and improves stability of the pangesture.

There may be several panning modes. When accelerometer navigation isenabled and the mobile device is angled below a certain value, atouch-and-drag gesture may have a different behavior. In that case,while the touch-and-drag gesture in the vertical direction may causepanning as described above with respect FIG. 17, a touch-and-draggesture in the horizontal direction may cause the virtual camera tolook-around. This is illustrated in FIGS. 19A-C.

FIG. 19A shows a diagram 1900 illustrating a mobile device 1904. Mobiledevice 1904 has an accelerometer that detects its angle relative togravity. When the angle of the mobile device is above a threshold a., auser can pan in all directions as illustrated in diagram 1930 in FIG.19B. When the of the mobile device is below a threshold a., atouch-and-grab gesture to the left and right does not pan, but causesthe virtual camera to look left and right as illustrated in diagram 1960in FIG. 19C. The virtual camera may look to the left and right bychanging a yaw value of the virtual camera.

CONCLUSION

Note that in the preceding description embodiments have for clarity beendescribed with respect to fingers making contact with a touch screen.However, any other object, such as a stylus, may be used as is known tothose of skill in the art.

The Summary and Abstract sections may set forth one or more but not allexemplary embodiments of the present invention as contemplated by theinventor(s), and thus, are not intended to limit the present inventionand the appended claims in any way.

The present invention has been described above with the aid offunctional building blocks illustrating the implementation of specifiedfunctions and relationships thereof. The boundaries of these functionalbuilding blocks have been arbitrarily defined herein for the convenienceof the description. Alternate boundaries can be defined so long as thespecified functions and relationships thereof are appropriatelyperformed.

The foregoing description of the specific embodiments will so fullyreveal the general nature of the invention that others can, by applyingknowledge within the skill of the art, readily modify and/or adapt forvarious applications such specific embodiments, without undueexperimentation, without departing from the general concept of thepresent invention. Therefore, such adaptations and modifications areintended to be within the meaning and range of equivalents of thedisclosed embodiments, based on the teaching and guidance presentedherein. It is to be understood that the phraseology or terminologyherein is for the purpose of description and not of limitation, suchthat the terminology or phraseology of the present specification is tobe interpreted by the skilled artisan in light of the teachings andguidance.

The breadth and scope of the present invention should not be limited byany of the above-described exemplary embodiments, but should be definedonly in accordance with the following claims and their equivalents.

What is claimed is:
 1. A computer-implemented method for navigating avirtual camera in a three dimensional environment on a mobile devicehaving a touch screen, comprising: receiving a user input indicatingthat an object has touched a first point on a touch screen of a mobiledevice and the object has been dragged to a second point on the touchscreen; receiving an orientation of the mobile device; determining afirst panning mode for a virtual camera in a three dimensionalenvironment on the mobile device when the orientation of the mobiledevice is below an angular threshold, wherein the three dimensionalenvironment includes a three dimensional model, and wherein the firstpanning mode corresponds to a mode in which the virtual camera staysstationary and the three-dimensional model is translated only when theuser input is up and down and in which a yaw value of the virtual camerachanges when the user input is left and right; and determining a secondpanning mode for the virtual camera in a three dimensional environmenton the mobile device when the orientation of the mobile device is abovethe angular threshold, wherein the second panning mode corresponds to amode in which the virtual camera stays stationary and thethree-dimensional model is translated in all directions based on theuser input; wherein the user input indicating that an object has toucheda first point on a screen of the mobile device and the object has beendragged to a second point on the touch screen causes a virtual camera tohave a different behavior when the first panning mode is determined thanwhen the second panning mode is determined.
 2. The computer-implementedmethod of claim 1, wherein receiving an orientation of the mobile devicecomprises receiving an angle of the mobile device relative to gravity.3. The computer-implemented method of claim 1, wherein: determining afirst panning mode occurs when the orientation of the mobile device isfacing downward; and determining a second panning mode occurs when theorientation of the mobile device is facing toward a horizon.
 4. Thecomputer-implemented method of claim 1, further comprising: moving athree dimensional model to follow movement of the object up or down onthe touch screen when the first panning mode is determined; and changingan orientation of the virtual camera according to movement of the objectleft or right on the touch screen when the first panning mode isdetermined.
 5. The computer-implemented method of claim 1, furthercomprising: moving a three dimensional model to follow movement of theobject up or down on the touch screen when the second panning mode isdetermined; and moving a three dimensional model to follow movement ofthe object left or right on the touch screen when the second panningmode is determined.
 6. The computer-implemented method of claim 1,further comprising: changing an orientation of the virtual cameraaccording to movement of the object left or right on the touch screenwhen the first panning mode is determined; and moving a threedimensional model to follow movement of the object left or right on thetouch screen when the second panning mode is determined.
 7. Thecomputer-implemented method of claim 1, wherein the three dimensionalmodel includes a three dimensional model of the Earth.